Search AI Products and News

Ant Group's Ling team recently published a preprint on arXiv titled "Every FLOP Matters: Scaling a 300-billion parameter Mixture-of-Experts LING model without high-end GPUs," detailing two novel large language models: Ling-Lite and Ling-Plus. These models incorporate several innovations enabling efficient training on low-performance hardware, significantly reducing training costs.

ByteDance's Doubao large model team recently announced a breakthrough in addressing key bottlenecks in Mixture-of-Experts (MoE) architecture, open-sourcing a significant optimization technology called COMET. This technology dramatically improves large model training efficiency, achieving a remarkable 1.7x speedup and a 40% reduction in training costs. Image Note: Image generated by AI, image licensing provider Midjourney. COMET has been deployed in ByteDance's multi-thousand-GPU cluster training, resulting in millions of GPU hours saved.